Formative Evaluation As the Saanich International Program Pre-Arrival Program is a pilot, the course designers feel that it is necessary to survey the students who complete Food, Culture and Recreation unit of the Welcome to Canada course to aid subsequent development of the program and to determine the unit’s effectiveness in advancing the program’s aims. Designers have used Gooler’s (1980) eight-step approach when developing this evaluation (Appendix ). Material for Appendix _. Below is an outline of the formative evaluation creation process followed by course designers. This development used Gooler’s (1980) eight-step approach to developing formative evaluation. Purpose[[#_msocom_1|[md1]]] - The evaluation’s main purpose is to determine how effectively the Welcome to Canada course’s Food, Culture and Recreation unit supports international students in preparing for the climate, culture and social mores of the country to which they are journeying. Audience – The target recipients of the evaluation’s results will be instructional designers involved in the program’s development, as well as administrators involved in the Saanich International Program and those at SIDES. Issues – Major questions include: · After receiving instruction can students correctly use knowledge of Canadian culture to help with their arrival in Canada? · Did students perceive the information in the unit to be helpful when they transitioned from their home countries to Canada? · On average how much time did the students spend on this unit? · In general do students feel this unit is important for students who are coming to Canada? · Is there a topic not covered by the unit, which students feel would be necessary to include? Resources – Because the information we’d like to glean is from the students themselves, and mostly qualitative[[#_msocom_2|[R2]]] (feelings-based rather than results-based), the evaluation will be conducted using a post-course survey. We will attach this to the online course unit as a quiz that the student will be required to complete before exiting the unit of study. Evidence – Stakeholders will review the results and draw their conclusions based on the number of similar answers to evaluation questions. As there is a negative consequence for not completing the evaluation (non-completion of unit), the results must be read with a degree of skepticism. Designers feel that the questions asked are non-personal enough to elicit answers that will be relatively objective and focused on the applicability of the unit’s material to the international student experience. Data Gathering Techniques – Designers will use an exit survey developed using the D2L quiz tool as their primary source of data –gathering. Analysis and Reporting– The D2L quiz tool has an immense amount of functions when it comes to analyzing data gleaned from its quizzes. D2L’s quiz tool is able to generate reports, graph responses using tables and present written comments in lists connected to the evaluation questions. Course designers will use these results to complete an executive summary of their findings. This summary will be presented at a stakeholder meeting or course review meeting.
Formative Evaluation: (Rose)
Pre-test: Mossison, Ross, Kalman and Kemp (2011) suggest that pre-tests are a valuable form of assessment as they provide instructional designers and instructors with useful information on what skills a student may already have, it can measure levels of mastery of skills and informs the learner of the subject matter that will be taught. The purpose for the pre-test for this unit is two-fold. Students need to demonstrate that they have minimum levels of English language skills in reading, writing and speaking. The second reason for the pre-test is to determine which competencies students have mastered, and to measure the degree of improvement after the instruction has been completed. The pre and post-test questions for this unit specifically address the learning objectives which address Canadian culture, food and recreation.
Summative Evaluation: Post-test: Elements of objective testing are used in the pre-test in the form of multiple choice questions, as well as true and false questions. Constructed-response questions are used in the post test. An essay question is used to determine if the learner was able to synthesize their learning, and to demonstrate their written English language skills. A rubic would be created and used to evaluate the writing (Morrison, et al, 2011 p. 325). The post-test has been designed to determine the learners’ improvement in performance.
Morrison, G., Ross, S., Kalman, H., and Kemp, J. (2011). Designing effective instruction 6th Ed. Hoboken:John Wiley and Sons, Inc.
Formative Evaluation
As the Saanich International Program Pre-Arrival Program is a pilot, the course designers feel that it is necessary to survey the students who complete Food, Culture and Recreation unit of the Welcome to Canada course to aid subsequent development of the program and to determine the unit’s effectiveness in advancing the program’s aims. Designers have used Gooler’s (1980) eight-step approach when developing this evaluation (Appendix ).
Material for Appendix _.
Below is an outline of the formative evaluation creation process followed by course designers. This development used Gooler’s (1980) eight-step approach to developing formative evaluation.
Purpose[[#_msocom_1|[md1]]] - The evaluation’s main purpose is to determine how effectively the Welcome to Canada course’s Food, Culture and Recreation unit supports international students in preparing for the climate, culture and social mores of the country to which they are journeying.
Audience – The target recipients of the evaluation’s results will be instructional designers involved in the program’s development, as well as administrators involved in the Saanich International Program and those at SIDES.
Issues – Major questions include:
· After receiving instruction can students correctly use knowledge of Canadian culture to help with their arrival in Canada?
· Did students perceive the information in the unit to be helpful when they transitioned from their home countries to Canada?
· On average how much time did the students spend on this unit?
· In general do students feel this unit is important for students who are coming to Canada?
· Is there a topic not covered by the unit, which students feel would be necessary to include?
Resources – Because the information we’d like to glean is from the students themselves, and mostly qualitative[[#_msocom_2|[R2]]] (feelings-based rather than results-based), the evaluation will be conducted using a post-course survey. We will attach this to the online course unit as a quiz that the student will be required to complete before exiting the unit of study.
Evidence – Stakeholders will review the results and draw their conclusions based on the number of similar answers to evaluation questions. As there is a negative consequence for not completing the evaluation (non-completion of unit), the results must be read with a degree of skepticism. Designers feel that the questions asked are non-personal enough to elicit answers that will be relatively objective and focused on the applicability of the unit’s material to the international student experience.
Data Gathering Techniques – Designers will use an exit survey developed using the D2L quiz tool as their primary source of data –gathering.
Analysis and Reporting– The D2L quiz tool has an immense amount of functions when it comes to analyzing data gleaned from its quizzes. D2L’s quiz tool is able to generate reports, graph responses using tables and present written comments in lists connected to the evaluation questions. Course designers will use these results to complete an executive summary of their findings. This summary will be presented at a stakeholder meeting or course review meeting.
Formative Evaluation: (Rose)
Pre-test:
Mossison, Ross, Kalman and Kemp (2011) suggest that pre-tests are a valuable form of assessment as they provide instructional designers and instructors with useful information on what skills a student may already have, it can measure levels of mastery of skills and informs the learner of the subject matter that will be taught. The purpose for the pre-test for this unit is two-fold. Students need to demonstrate that they have minimum levels of English language skills in reading, writing and speaking. The second reason for the pre-test is to determine which competencies students have mastered, and to measure the degree of improvement after the instruction has been completed. The pre and post-test questions for this unit specifically address the learning objectives which address Canadian culture, food and recreation.
Summative Evaluation:
Post-test:
Elements of objective testing are used in the pre-test in the form of multiple choice questions, as well as true and false questions. Constructed-response questions are used in the post test. An essay question is used to determine if the learner was able to synthesize their learning, and to demonstrate their written English language skills. A rubic would be created and used to evaluate the writing (Morrison, et al, 2011 p. 325). The post-test has been designed to determine the learners’ improvement in performance.
Morrison, G., Ross, S., Kalman, H., and Kemp, J. (2011). Designing effective instruction 6th Ed.
Hoboken:John Wiley and Sons, Inc.